MathSymb performance measurements
overview
want to understand this report?
mathExpand (True)/ExprAF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.537933215098387e-6 | 1.5678736867838272e-6 | 1.6691325374261059e-6 |
| Standard deviation | 3.426878995488073e-8 | 1.3083521423473795e-7 | 2.431064971541719e-7 |
Outlying measurements have severe (0.8274947520457279%) effect on estimated standard deviation.
mathExpand (True)/ExprAF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.3601243628684797e-6 | 3.370973573892728e-6 | 3.38855737679063e-6 |
| Standard deviation | 3.657744648650429e-8 | 4.669963641506512e-8 | 6.000061848269316e-8 |
Outlying measurements have moderate (0.10689678841347112%) effect on estimated standard deviation.
mathExpand (True)/ExprAF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.32368426998138e-6 | 7.369431415238519e-6 | 7.389113834965716e-6 |
| Standard deviation | 5.8922579734320165e-8 | 8.074975765427834e-8 | 9.51877150099583e-8 |
Outlying measurements have slight (6.208373518253141e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprAF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.5681990595309483e-5 | 1.5721854037845653e-5 | 1.5777155328081436e-5 |
| Standard deviation | 9.293722867867907e-8 | 1.2872106569236873e-7 | 1.5235693036620546e-7 |
Outlying measurements have no (7.2988754325262264e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprAF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.4657330193082014e-5 | 3.476069883935099e-5 | 3.4900971507547554e-5 |
| Standard deviation | 2.510479198117248e-7 | 3.1456110688453106e-7 | 4.0851359921563005e-7 |
Outlying measurements have no (8.263888888889001e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprAF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.566618205201865e-5 | 7.590702229952834e-5 | 7.605703697142418e-5 |
| Standard deviation | 4.7107562059839585e-7 | 5.914520811533234e-7 | 8.072068589812974e-7 |
Outlying measurements have no (9.522928994082474e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprAF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.5969812143360406e-4 | 1.6061864290209125e-4 | 1.6135105449005183e-4 |
| Standard deviation | 1.3417625072776033e-6 | 1.6216035679614883e-6 | 2.000759555954877e-6 |
Outlying measurements have slight (1.1109708370155055e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprBF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.64667118872896e-7 | 5.715680928709941e-7 | 5.751634441701591e-7 |
| Standard deviation | 1.2023892023876439e-8 | 1.5272551695548433e-8 | 1.9514368363645005e-8 |
Outlying measurements have moderate (0.3486292911251059%) effect on estimated standard deviation.
mathExpand (True)/ExprBF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4386797367258054e-6 | 1.4443426132057705e-6 | 1.4535601312914677e-6 |
| Standard deviation | 1.426755497854398e-8 | 1.7252817366074455e-8 | 2.15655095893084e-8 |
Outlying measurements have slight (8.689358310186186e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprBF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.619805585841152e-6 | 7.648702476422027e-6 | 7.672004714595537e-6 |
| Standard deviation | 5.323046320351763e-8 | 6.880561478024444e-8 | 8.482666826728879e-8 |
Outlying measurements have no (6.57865883075311e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprBF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.671753411931134e-5 | 1.6773720281833276e-5 | 1.6852009760744934e-5 |
| Standard deviation | 1.18334653289465e-7 | 1.6093523754872953e-7 | 2.0466603763605866e-7 |
Outlying measurements have no (7.352537722907879e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprBF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.601852180603289e-5 | 3.619652030290175e-5 | 3.627656880079875e-5 |
| Standard deviation | 2.238803827145018e-7 | 2.9215851385924906e-7 | 3.7685673403465515e-7 |
Outlying measurements have no (8.332744862650749e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprBF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.798291216085096e-5 | 7.835780445717603e-5 | 7.862562267930853e-5 |
| Standard deviation | 7.166451938964129e-7 | 8.57750580586413e-7 | 1.0682396142136983e-6 |
Outlying measurements have no (9.614478273164247e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprBF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6600987572330429e-4 | 1.6664226842878356e-4 | 1.6718315270870107e-4 |
| Standard deviation | 1.1878711374086048e-6 | 1.4533858801469674e-6 | 1.829068903286692e-6 |
Outlying measurements have slight (1.1234504132231427e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprCF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.783036019999854e-7 | 3.811080164966407e-7 | 3.897275097255769e-7 |
| Standard deviation | 7.807653138481289e-9 | 1.2421522934357154e-8 | 2.0267402111350198e-8 |
Outlying measurements have moderate (0.45185146716049546%) effect on estimated standard deviation.
mathExpand (True)/ExprCF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.120014555400299e-7 | 8.207527707110646e-7 | 8.347356696604307e-7 |
| Standard deviation | 2.3676070380764142e-8 | 2.7978193313518475e-8 | 3.482265389992162e-8 |
Outlying measurements have moderate (0.45427259162972344%) effect on estimated standard deviation.
mathExpand (True)/ExprCF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6374790054387167e-6 | 1.659838826019068e-6 | 1.6996145179472186e-6 |
| Standard deviation | 4.587857193569851e-8 | 6.396538561833838e-8 | 9.046452513254278e-8 |
Outlying measurements have severe (0.501062842246504%) effect on estimated standard deviation.
mathExpand (True)/ExprCF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.3869457363791455e-6 | 3.4369362291953746e-6 | 3.5422686146819744e-6 |
| Standard deviation | 1.3710825677028786e-7 | 1.6833688150917535e-7 | 2.202374226087778e-7 |
Outlying measurements have severe (0.5962286992067586%) effect on estimated standard deviation.
mathExpand (True)/ExprCF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.635393300207579e-6 | 6.687799835430544e-6 | 6.7208861140797716e-6 |
| Standard deviation | 9.627544731217754e-8 | 1.2703358297658308e-7 | 1.682994509045358e-7 |
Outlying measurements have moderate (0.16765598640307952%) effect on estimated standard deviation.
mathExpand (True)/ExprCF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3271170311176908e-5 | 1.332623045781871e-5 | 1.3429569418671396e-5 |
| Standard deviation | 1.4420810775509695e-7 | 1.8166450112863698e-7 | 2.3031172527379513e-7 |
Outlying measurements have slight (8.823419123275354e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprCF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.727200400853656e-5 | 2.7777320963637472e-5 | 2.8996673958876017e-5 |
| Standard deviation | 6.166554501216519e-7 | 2.0623017053622166e-6 | 3.7258525230027973e-6 |
Outlying measurements have severe (0.7237159392935678%) effect on estimated standard deviation.
mathExpand (True)/ExprDF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.0648045852065436e-7 | 4.108668735791744e-7 | 4.1814926788213586e-7 |
| Standard deviation | 1.1507815343667513e-8 | 1.3727116027925613e-8 | 1.6801504159109082e-8 |
Outlying measurements have moderate (0.46347244449411645%) effect on estimated standard deviation.
mathExpand (True)/ExprDF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.5457926269848775e-6 | 3.5597574061486624e-6 | 3.572817806677653e-6 |
| Standard deviation | 2.803671109339395e-8 | 3.4567247816770184e-8 | 4.254519788842851e-8 |
Outlying measurements have no (7.753535167070919e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprDF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.5328817654080686e-6 | 3.5562488827581364e-6 | 3.5745664850507046e-6 |
| Standard deviation | 3.593094959087783e-8 | 4.588686658689249e-8 | 6.086510277113019e-8 |
Outlying measurements have slight (9.113331845088492e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprDF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.3652842201564336e-5 | 4.38488702705761e-5 | 4.409245467945415e-5 |
| Standard deviation | 2.2780953249499223e-7 | 6.263496444555613e-7 | 1.0695689487194341e-6 |
Outlying measurements have slight (8.192140375259607e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprDF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2268802708799073e-4 | 1.228524919468981e-4 | 1.2329449308283075e-4 |
| Standard deviation | 4.953506084439198e-7 | 6.387670086594569e-7 | 8.644268502944128e-7 |
Outlying measurements have slight (1.0525124490719774e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprDF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.6394961974198134e-4 | 2.644139746351399e-4 | 2.650843362514769e-4 |
| Standard deviation | 1.1460738900622807e-6 | 1.469982345573898e-6 | 1.7729999903509268e-6 |
Outlying measurements have slight (1.2497997115846815e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprDF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.546518770036321e-4 | 2.55222886864684e-4 | 2.557011702060015e-4 |
| Standard deviation | 1.143832515948854e-6 | 1.4110800774880006e-6 | 1.8310075869080042e-6 |
Outlying measurements have slight (1.234375e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprEF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3444700944067876e-7 | 1.3658129401945998e-7 | 1.3860260200148532e-7 |
| Standard deviation | 3.5241879438580154e-9 | 4.835108834495239e-9 | 6.24529617890697e-9 |
Outlying measurements have severe (0.5170335051397322%) effect on estimated standard deviation.
mathExpand (True)/ExprEF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.202074155182444e-7 | 4.2498403573593903e-7 | 4.30053026478764e-7 |
| Standard deviation | 1.1392470675435062e-8 | 1.470663934819259e-8 | 1.9628098305507936e-8 |
Outlying measurements have moderate (0.4799994215836105%) effect on estimated standard deviation.
mathExpand (True)/ExprEF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.039057629728634e-6 | 6.0609227655030505e-6 | 6.103352575815759e-6 |
| Standard deviation | 6.72132953973145e-8 | 8.968014717978809e-8 | 1.3103078187377076e-7 |
Outlying measurements have moderate (0.10929379046200408%) effect on estimated standard deviation.
mathExpand (True)/ExprEF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.194166086750944e-6 | 4.218377522628029e-6 | 4.2450676053638315e-6 |
| Standard deviation | 3.9347426879583337e-8 | 5.5136362646600316e-8 | 6.58546150054416e-8 |
Outlying measurements have slight (9.27530635451296e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprEF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.619616117766857e-5 | 1.6256155438026686e-5 | 1.6367779612607667e-5 |
| Standard deviation | 1.605563343763034e-7 | 2.0418582092597788e-7 | 2.661017736232666e-7 |
Outlying measurements have slight (7.661081248753258e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprEF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.665222860815627e-6 | 4.6834393795652485e-6 | 4.708721215761062e-6 |
| Standard deviation | 3.423308450058931e-8 | 4.720471205169475e-8 | 6.104684143883079e-8 |
Outlying measurements have slight (5.028977637170204e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprEF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.27847997108489e-5 | 9.296096048855004e-5 | 9.31304599561638e-5 |
| Standard deviation | 4.173311287137116e-7 | 5.082967741532132e-7 | 7.322137357029924e-7 |
Outlying measurements have no (9.900000000000138e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprFF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0096614005261764e-6 | 1.0230622827563206e-6 | 1.0361967327802208e-6 |
| Standard deviation | 3.513482461550239e-8 | 4.2681148534565164e-8 | 5.320986180506144e-8 |
Outlying measurements have severe (0.5526387484821008%) effect on estimated standard deviation.
mathExpand (True)/ExprFF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.8949470843814917e-6 | 1.9035671343094684e-6 | 1.913461912157655e-6 |
| Standard deviation | 1.6096631590261623e-8 | 2.0553151460139165e-8 | 2.5815314073814955e-8 |
Outlying measurements have slight (7.363196275751518e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprFF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0447419280476812e-5 | 1.0469839701093744e-5 | 1.0494569555855629e-5 |
| Standard deviation | 4.8593870559222206e-8 | 5.932704156161069e-8 | 7.00537208419584e-8 |
Outlying measurements have no (6.896219135802524e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprFF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.020535659173721e-6 | 7.0369218670231944e-6 | 7.056530735694144e-6 |
| Standard deviation | 3.1626975940066716e-8 | 4.862966019936313e-8 | 6.537587188917773e-8 |
Outlying measurements have no (6.5356648199441335e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprFF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6765195758362163e-5 | 1.6811701378376224e-5 | 1.6890642145899242e-5 |
| Standard deviation | 9.303137655888e-8 | 1.4129824465354223e-7 | 2.1658340300337316e-7 |
Outlying measurements have no (7.352537722908018e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprFF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.340128510042527e-5 | 3.3471626472044845e-5 | 3.354579482857291e-5 |
| Standard deviation | 1.2200619446057637e-7 | 1.8314063439204103e-7 | 2.3951319289886844e-7 |
Outlying measurements have no (8.196161464380892e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprFF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.353254655845135e-5 | 9.373920253784671e-5 | 9.393143859724589e-5 |
| Standard deviation | 4.051630351172079e-7 | 5.058039032729412e-7 | 5.869047274802179e-7 |
Outlying measurements have no (9.899999999999872e-3%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.